Protein methyltransferases (PMTs) are a promising target class in oncology and other disease areas. They are composed of SET domain methyltransferases and structurally unrelated Rossman-fold enzymes that include protein arginine methyltransferases (PRMTs). In the absence of a well-defined medicinal chemistry tool-kit focused on PMTs, most current inhibitors were identified by screening large and diverse libraries of leadlike molecules. So far, no successful fragment-based approach was reported against this target class. Here, by deconstructing potent PRMT inhibitors, we find that chemical moieties occupying the substrate arginine-binding site can act as efficient fragment inhibitors. Screening a fragment library against PRMT6 produced numerous hits, including a 300 nM inhibitor (ligand efficiency of 0.56) that decreased global histone 3 arginine 2 methylation in cells, and can serve as a warhead for the development of PRMT chemical probes.